Papers by Zhen Leng Thai
Between a Rock and a Hard Place: The Tension Between Ethical Reasoning and Safety Alignment in LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) safety alignment predominantly operates on a binary assumption that requests are either safe or unsafe. |
| Approach: | They propose a methodology that embeds harmful requests within ethical framings to exploit this vulnerability. |
| Outcome: | The proposed framework achieves high success rates by exploiting model's own ethical reasoning to frame harmful actions as morally necessary compromises. |
Cost-Optimal Grouped-Query Attention for Long-Context Modeling (2025.emnlp-main)
Copied to clipboard
Yingfa Chen, Yutong Wu, Chenyang Song, Zhen Leng Thai, Xingyu Shen, Xu Han, Zhiyuan Liu, Maosong Sun
| Challenge: | Current GQA configurations overlook how context length influences inference cost . |
| Approach: | They propose a recipe for deriving cost-optimal GQA configurations that decouple the total head size from the hidden size and allow more flexible control over attention FLOPs. |
| Outcome: | The proposed configurations reduce memory usage and FLOPs by more than 50% compared to Llama-3's GQA, with *no degradation in model capabilities*. |
StateX: Enhancing RNN Recall via Post-training State Expansion (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that RNNs with large recurrent states are expensive to train . however, the ability to recall contextual information from long contexts is underperforms them in certain aspects. |
| Approach: | They propose a framework that expands the states of pre-trained RNNs by scaling them up to 1.3B . they use a recurrent architecture that compresses contextual information into a fixedsize state . |
| Outcome: | Experiments on models with up to 1.3B parameters show that StateX expands state sizes without incurring high post-training costs or compromising other capabilities. |